Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(docker): support remote multi-platform compilation on ci #5204

Closed
wants to merge 12 commits into from
Closed

feat(docker): support remote multi-platform compilation on ci #5204

wants to merge 12 commits into from

Conversation

oguzkaganozt
Copy link
Contributor

@oguzkaganozt oguzkaganozt commented Sep 9, 2024 •
edited
Loading

Description

Support remote multi-platform compilation in order to push images in a single manifest with multiple architectures.

Tests performed

https://github.com/autowarefoundation/autoware/actions/runs/10828791104

Effects on system behavior

Not applicable.

Interface changes

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@oguzkaganozt oguzkaganozt self-assigned this Sep 9, 2024
@oguzkaganozt oguzkaganozt changed the title feat(docker): Support cross compilation on ci feat(docker): support cross compilation on ci Sep 9, 2024
youtalk
youtalk requested changes Sep 9, 2024
View reviewed changes
Copy link
Member

@youtalk youtalk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! But the CIs were failed. Please pass them then I will review this.
https://github.com/autowarefoundation/autoware/actions/runs/10777587669

@oguzkaganozt oguzkaganozt reopened this Sep 11, 2024
@oguzkaganozt oguzkaganozt changed the title feat(docker): support cross compilation on ci feat(docker): support remote multi-platform compilation on ci Sep 11, 2024
@oguzkaganozt oguzkaganozt marked this pull request as draft September 11, 2024 10:38
xmfcx
xmfcx reviewed Sep 11, 2024
View reviewed changes
docker/Dockerfile Outdated Show resolved Hide resolved
@oguzkaganozt oguzkaganozt marked this pull request as ready for review September 11, 2024 16:00
oguzkaganozt and others added 8 commits September 11, 2024 19:38
@oguzkaganozt
enable remote node builder for multi-arch builds
c9f50f4 
Signed-off-by: Oguz Ozturk <[email protected]>
@pre-commit-ci @oguzkaganozt
style(pre-commit): autofix
6973f5d
@oguzkaganozt
add remote secrets
c6ad451 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
b5424db 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
543092f 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
6462b95 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
f96dfea 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
e1ef328 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
51dd0d6 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
8d22587 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
e6649e8 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
.
9629605 
Signed-off-by: Oguz Ozturk <[email protected]>
@oguzkaganozt
Copy link
Contributor Author

oguzkaganozt commented Sep 12, 2024 •
edited
Loading

Updated the PR to its latest state and here is the test run, it can now build arm64 images remotely on aws runner in parallel to local amd64 build.

If it will not exceed the 6 hours limit of standart job execution time than it will produce multi-architecture images on a single workflow run.

If it exceeds the job execution time then we have only two options remaining to achieve multi-architecture images:
a) Upgrade aws runner
b) Upgrade github plan to have larger limits.

@mitsudome-r @youtalk @xmfcx

@mitsudome-r
Copy link
Member

mitsudome-r commented Sep 13, 2024 •
edited
Loading

@oguzkaganozt It seems like pre-commit is failing. Could you fix your PR so that it passes the CI? (you can also test locally by running pre-commit run -a.

Also your test seems to be failing as well.

@oguzkaganozt oguzkaganozt mentioned this pull request Sep 19, 2024
Merged
4 tasks
@oguzkaganozt
Copy link
Contributor Author

After investigating the need of combined workflows which eventually leads to 6-hour runtime constraint on github runners, I am suggesting this solution which will produce the similar result without the need for upgrades.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@youtalk youtalk Awaiting requested review from youtalk youtalk is a code owner

@xmfcx xmfcx Awaiting requested review from xmfcx

Assignees

@oguzkaganozt oguzkaganozt

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@oguzkaganozt @mitsudome-r @youtalk @xmfcx